Computer viruses and other malicious programs which are spread through the Internet and local networks and through portable media continue to present an ongoing challenge to the public. Perhaps the most prominent of these are the so-called exploits, which can take the form of code fragments or commands that take advantage of vulnerabilities in software. Such vulnerabilities are caused by mistakes made when programming or designing applications. With the growth of the number and complexity of applications, the number of vulnerabilities inevitably grows as well; writers of malicious programs actively take advantage of this, already using not only single exploits but whole sets of them, designed for multiple programs and for their various versions. Exploits can be designed for specific operating systems, browsers, websites, etc.
In order to resolve the issue of countering the exploits, software manufacturers constantly produce updates of their applications in order to cover for known vulnerabilities, while manufacturers of operating systems and anti-virus companies use various methods for fighting known exploits.
One of the anti-exploit methods involves using a security function named DEP (Data Execution Prevention), which marks certain parts of memory as being intended to hold only data, which the NX bit enabled processor then understands as non-executable. This technology is being successfully used for protection from buffer overflow attacks, when the application writes data beyond the buffer allocated in the memory.
A related technology has been proposed for detecting possible malicious code that gets unpacked at run-time, such as the approach disclosed in U.S. Pat. No. 8,104,089. This approach injects a packing manager into a process, or installs it in the operating system's kernel. The packing manager uses memory page write and execute permissions to track when a program attempts to first write to, and then execute code from, a memory page. This approach uses the operations of the unpacker to restore the packed code from its packed state, but ensures that the code cannot be subsequently executed without being checked for malware. This technique deals particularly with packers, but it does not address the diversity of operations that are used by exploits, which can introduce malicious code in a variety of ways, not just through unpacking of the code.
Another technology, named ASLR (Address Space Layout Randomization), randomly arranging the positions of key data areas, usually including the base of the executable and position of libraries, heap, and stack in the address space of the process; this significantly complicates the exploitation of some vulnerabilities.
It should be pointed out that the above-mentioned technologies allow to successfully fight most of today's exploits, but in a number of cases, it is not possible to track the launch of the latest exploit variants, as exploit creators always look for newer ways to bypass the existing protection technologies. A practical and efficient solution is needed to address these challenges.